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INTRODUCTION 

It  is  a  common  assertion  that  humans  have  ’’mental  models”  of 
the  systems  with  which  they  interact.  In  fact,  it  is  difficult 
to  explain  many  aspects  of  human  behavior  without  resorting  to  a 
construct  such  as  mental  models  [Conant  and  Ashby,  1970]. 
However,  acceptance  of  the  logical  necessity  of  mental  models 
does  not  eliminate  conceptual  and  practical  difficulties;  it 
simply  raises  a  whole  new  set  of  finer-grained  issues. 

For  example,  what  forms  do  mental  models  take?  How  does  the 
form  affect  the  usage  of  the  models?  Is  guidance  in  the  use  of 
models  as  important  as  their  form?  How  can  and  should  designers 
and  trainers  attempt  to  affect  humans'  mental  models?  These 
really  are  not  new  questions.  However,  as  is  discussed  later, 
once  they  are  expressed  in  terms  of  the  concept  of  mental  models, 
they  tend  to  be  dealt  with  somewhat  differently. 

Further,  despite  many  sweeping  claims  in  the  contemporary 
literature,  available  answers  to  the  above  questions  are  rather 
inadequate.  There  are  prospects  for  improving  this  situation. 
However,  there  also  are  limits;  the  ’’black  box”  of  human  mental 
models  will  never  be  completely  transparent.  This  paper 
considers  these  prospects  and  limits. 

To  place  the  arguments  advanced  in  this  paper  in 
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perspective^  several  points  of  view  with  regard  to  mental  models 
are  first  reviewed.  Alternative  definitions,  purposes,  and 
taxonomies  are  discussed  in  the  context  of  a  variety  of 
behavioral  domains.  This  leads  to  a  discussion  of  differences 
among  domains,  particularly  in  terms  of  methods  for  identifying 
the  form,  structure,  parameters,  etc.  of  mental  models.  From 
this  discussion  emerges  a  key  set  of  issues,  which  initially  are 
discussed  in  general.  Discussion  then  focuses  on  issues 
specifically  associated  with  instruction  (i.e.,  fostering  the 
creation  of  mental  models).  Finally,  fundamental  limits  in  the 
search  for  mental  models  are  considered. 

DEFINITIONS 

While  the  phrase  "mental  models"  is  ubiquitous  in  the 
literature,  there  are  surprisingly  few  explicit  definitions 
provided.  Thi3  most  likely  reflects  the  extent  to  which  the 
concept  has  come  to  be  completely  acceptable  on  an  almost 
intuitive  basis.  Nevertheless,  it  is  interesting  to  consider  the 
few  formal  definitions  that  have  been  espoused. 

The  manual  control  community  has  traditionally  focused  on 
skilled,  psychomotor  performance.  More  recently,  the  term 
"manual"  is  giving  way  to  "supervisory"  in  recognition  of  the 
fact  that  the  human's  role  is  increasingly  becoming  one  of 
monitoring  automatically-controlled  systems  for  the  purpose  of 
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detecting,  diagnosing,  and  compensating  for  system  failures 
[Sheridan  and  Johannsen,  1976;  Rasmussen  and  Rouse,  1981].  In 
reviewing  the  use  of  the  concept  of  mental  models  in  this  domain, 
Veldhuyzen  and  Stassen  [1977]  conclude  that  a  human's  mental 
model  includes  knowledge  about  the  system  to  be  controlled, 
knowledge  about  the  properties  of  disturbances  likely  to  act  on 
the  system,  and  knowledge  about  the  criteria,  strategies,  etc. 
associated  with  the  control  task.  In  a  recent,  and  more 
circumspect,  discussion  of  research  in  this  area,  Wickens  [1984] 
refers  to  the  concept  of  a  mental  model  as  a  "hypothetical 
construct"  to  account  for  human  behavior  in  sampling,  scanning, 
planning,  etc.  Jagacinski  and  Miller  [1978],  also  working  in 
manual  control,  define  mental  models  as  special  cases  of 
"schema,"  a  fairly  well-accepted  psychological  notion  of  how 
skilled  performance  is  organized  (see  Wickens  [1984]). 

While  the  manual  control  community  has  been  blithely  using 
the  mental  models  concept  for  at  least  twenty  years,  cognitive 
psychology  has  only  recently  embraced  this  notion.  This 
acceptance  is  clearest  in  the  area  of  "cognitive  science,"  which 
is  basically  the  result  of  a  liaison  between  cognitive  psychology 
and  computer  science  or  artificial  intelligence.  This  relatively 
new  community  of  researchers  has  recently  produced  an  edited  book 
on  mental  models  [Gentner  and  Stevens,  1983]. 


In  contrast  to  manual  and  supervisory  control  where  mental 
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models  serve  as  assumptions  which  allow  calculations  of  expected 
control  performance,  research  in  cognitive  science  tends  to  focus 
directly  on  mental  models,  particularly  in  terms  of  the  ways  in 
which  humans  understand  systems.  Norman  t 1983 3  characterizes 
this  understanding  as  messy,  sloppy,  incomplete,  and  indistinct 
knowledge  structures.  Lehner  and  his  colleagues  [19843  have 
asserted  that  humans'  mental  models  of  a  particular  class  of 
computer  programs  (i.e.,  expert  systems)  include  understanding 
that:  1)  the  program's  knowledge  is  encoded  in  rules,  2)  rules 
are  organized  in  the  program  in  terms  of  a  network  of 
relationships,  and  3)  explanatory  traces  of  program  behavior 
involve  chaining  along  this  network.  Definitions  that  emphasize 
somewhat  narrower  behavioral  domains  include  topologies  of  device 
models  [Brown  and  deKleer,  1981 5  deKleer  and  Brown,  19833  and 
collections  of  autonomous  objects  [Williams,  et  al.,  19833. 
Thus,  it  can  be  seen  that  definitions  within  the  cognitive 
science  community  range  from  broad  and  intentionally  amorphous 
generalizations  to  specific  and  somewhat  esoteric  constructs. 

A  very  significant  difficulty  with  the  phrase  "mental 
models"  involves  how  one  should  differentiate  this  concept  from 
that  of  "knowledge"  in  general.  Does  this  phrase  reflect  the 
common  tendancies  of  young  sciences  to  re-label  everyday 
phenomena?  Certainly  cognitive  science  and  especially  artificial 
intelligence  appear  to  have  penchants  for  coining  terminology. 
Nevertheless,  in  this  ease,'  it  appears  to  be  reasonable  to  employ 
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the  concept  of  mental  models  to  connote  special  types  of 
knowledge.  This  becomes  clear  when  one  considers  the  purposes 
that  mental  models  are  supposed  to  serve. 

PURPOSES 

The  above  discussion  tended  to  emphasize  the  differences  in 
perspectives  of  researchers  in  manual/supervisory  control  and 
cognitive  science.  These  differences  in  definitions  and 
terminology  are  considerably  lessened  once  one  considers 
purposes . 

Veldhuyzen  and  Stassen  [1977],  in  their  review  of  the  use  of 
the  mental  model  concept  in  manual  control,  conclude  that  mental 
models  provide  the  basis  for  estimating  the  "state"  of  the  system 
(i.e.,  estimating  state  variables  that  are  not  directly 
displayed),  developing  and  adopting  control  strategies,  selecting 
proper  control  actions,  determining  whether  or  not  actions  led  to 
desired  results,  and  understanding  unexpected  phenomena  that 
occur  as  the  task  progresses.  This  conclusion,  in  effect, 


asserts 

that  mental  models 

are  the 

basis  for 

all 

aspects 

of 

manual 

control.  Such  a 

sweeping 

assertion 

can 

lead  one 

to 

surmise  that  "mental  models"  are  synonymous  with  "knowledge"  in 
general. 


In  fact,  Veldhuyzen  and  Stassen  appear  to  be  correct  in  the 
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sense  that  the  phrase  mental  models  ^is  used  in  this  way, 
especially  in  the  manual/supervisory  control  community.  However, 
this  is  not  the  way  the  phrase  should  be  used.  More  precision  is 
needed;  otherwise,  there  is  a  great  risk  that  the  result  of 
research  in  this  area  will  simply  be  that,  “humans  have  to  know 
something  in  order  to  perform  their  tasks.”  Clearly,  this  result 
will  not  be  a  great  stride  for  science. 

Rasmussen  [1979,  1983],  also  working  within  the  domain  of 
supervisory  control,  limits  the  range  of  purposes  of*  mental 
models.  He  asserts  that  mental  models  are  for  predicting  future 
events,  finding  causes  of  observed  events,  and  determining 
appropriate  actions  to  cause  changes  [Rasmussen,  1979].  He  also 
includes  the  use  of  mental  models  for  performing  "internal” 
experiments  [Rasmussen,  1983],  or  what  physicists  refer  to  as 
"thought"  or  "Gedanken"  experiments  [Zukav,  1979]. 

Alexander  [1964]  discusses  the  "mental  pictures"  employed  by 
engineering  and  architectural  designers.  These  pictures  are 
defined  quite  broadly  in  terms  of  contexts  (problem  definitions) 
and  forms  (alternative  solutions).  Hence,  the  purposes  of 
designers*  mental  pictures  or  models  are  viewed  as  much  more 
encompassing  than  the  models  discussed  in  the  supervisory  control 
arena.  This  difference  in  scope  most  likely  reflects  inherent 
differences  between  open-ended  tasks  such  as  design  and 
well-defined  tasks  like  supervisory  control. 
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Within  the  cognitive  science  domain,  Williams  and  his 
colleagues  [1983]  claim  the  purposes  of  mental  models  to  be 
predicting  and  explaining  system  behavior  and  serving  as  mnemonic 
devices  for  remembering  relationships  and  events.  Evidencing  a 
more  traditional  psychological  point  of  view,  Wickens  [1984] 
reports  that  mental  models  are  constructs  used  by  researchers  to 
explain  display  sampling  and  scanning,  formulating  of  plans,  and 
translating  of  goals  into  actions.  He  also  suggests  that  mental 
models  are  sources  of  humans'  expectations. 

The  intersection  of  the  various  points  of  view  outlined  in 
this  section  leads  to  a  fairly  clear  set  of  purposes  for  mental 
models.  The  common  themes  are  describing,  explaining .  and 
predicting.  regardless  of  whether  the  human  is  performing 
internal  experiments,  scanning  displays,  or  executing  control 
actions.  These  three  terms  can  be  combined  with  a  modification 
of  Rasmussen ' s . taxonomy  of  mental  models  [Rasmussen,  1979]  to 
yield  the  integrated  view  of  the  purposes  of  mental  models  shown 
in  Figure  1 . 

Based  on  this  figure,  a  functional  definition  of  mental 
models  can  be  proposed:  mental  models  are  the  mechanisms  whereby 
humans  are  able  to  generate  descriptions  of  system  purpose  and 
form,  explanations  of  system  functioning  and  observed  system 
states,  and  predictions  of  future  system  states.  It  is  important 
to  emphasize  that  this  definition  does  not  differentiate  between 


PURPOSE 


WHY  A  SYSTEM  EXISTS 


DESCRIBING 


EXPLAINING 


PREDICTING 


FUNCTION 


STATE 


FORM 


HOW  A  SYSTEM  OPERATES 


WHAT  A  SYSTEM  IS  DOING 


WHAT  A  SYSTEM  LOOKS  LIKE 


Figure  1.  PURPOSES  OF  MENTAL  MODELS 
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knowledge  that  is  simply  retrieved  and  knowledge  that  involves 
some  type  of  calculation.  Thus,  humans'  mental  models  are  not 
necessarily  computational  models. 

It  was  noted  earlier  that  a  models  =  knowledge  definition 
should  be  avoided  if  the  mental  models  construct  is  to  have  any 
real  utility.  The  above  definition  does  not  eliminate  this 
problem,  which  serves  to  underscore  the  possibly  marginal  value 
of  the  construct.  Nevertheless,  the  proposed  definition  does 
specify  particular  types  of  knowledge  and  the  purposes  for  which 
this  knowledge  is  used.  This  level  of  specificity  is  sufficient 
to  enable  a  meaningful  inquiry  into  the  nature  of  mental  models. 

IDENTIFICATION 

Given  the  above  functional  definition  of  mental  models,  one 
can  then  reasonably  consider  how  these  mechanisms  might  be 
identified.  In  other  words,  what  forms,  structures,  parameters, 
etc.  are  associated  with  mental  models  of  particular  individuals 
for  specific  task  situations?  There  are  a  variety  of  approaches 
to  these  types  of  question. 

Inferring  Characteristics  Via  Empirical  Study 

Perhaps  the  most  traditional  approach  to  the  study  of  mental 
models  is  the  use  of  experimental  methods  to  infer  the 


characteristics  of  models.  This  approach  is  the  stock  in  trade 
of  experimental  psychology.  An  excellent  example  of  this 
approach  is  the  work  of  Kessel  and  Wickens  [1982]  who  studied  the 
impact  of  training  (manual  control  vs.  monitoring  of  automatic 
control)  on  subsequent  monitoring  performance.  They  found  that 
the  cue  utilization  abilities  fostered  by  manual  control  training 
were  more  successfully  transferred  to  subsequent  monitoring 
performance  than  training  based  on  monitoring  of  automatic 
control.  Despite  the  fact  that  proprioceptive  channels  (due  to 
control  stick  movements )•  were  no  longer  available  in  the  transfer 
conditions,  manual  training  was  clearly  superior.  Based  on  this 
finding,  the  investigators  inferred  that  the  mental  models 
developed  in  the  two  conditions  were  different  in  that  the  type 
of  information  employed  in  monitoring  depended  on  the  type  of 
training. 

While  inference  via  empirical  study  provides  evidence  for 
affects  of  various  independent  variables  on  characteristics  of 
mental  models,  these  types  of  result  provide,  at  best,  only 
indirect  insights  into  the  form  (e.g.,  spatial  vs.  verbal)  and 
structure  (e.g.,  hierarchical  vs.  planar)  of  mental  models. 
This  is  due  to  the  likelihood  that  access  and  manipulation  of 
models  are  confounded  with  perception  and  response  execution; 
interaction  among  these  three  stages  of  information  processing 
can  limit  the  precision  of  conclusions. 
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Empirical  Modeling 

In  situations  where  perception  and  response  execution  are 
unlikely  to  interact  with  model  manipulation,  empirical  modeling 
may  be  possible.  This  approach  involves  algorithmically 
identifying  the  relationship  between  humans'  observations  and 
subsequent  actions.  If  it  can  be  assumed  that  humans  actually 
perceive  what  is  displayed  and  response  execution  is  very  simple, 
then  techniques  such  as  regression  can  be  used  to  identify 
input-output  relationships.  From  these  relationships,  the 
structure  and  parameters  of  mental  models  can  be  inferred. 
Jagacinski  and  Miller  [1978]  employed  this  approach  for  a 
"bang-bang"  time-optimal  manual  control  task  where  regression  on 
subjects'  "switching  curves"  allowed  inferences  about  mental 
models.  Several  investigators  have  studied  the  relationships 
between  humans'  explicit  predictions  of  future  system  states  and 
currently  displayed  states,  using  regression  or  time-series 
models  to  identify  input-output  relationships  [Rouse,  1977;  van 
Bussel,  1980;  van  Heusden,  1980],  All  four  of  the  above  studies 
resulted  in  hypothesized  mental  models  that  differed 
systematically  from  the  "true"  model  of  the  system  involved. 

It  is  worth  noting  that  related  approaches  have  been 
employed  in  a  variety  of  studies  of  human  judgement.  Anderson's 
"cognitive  algebra"  and  Hammond's  "policy  capturing"  are  two 
notable  examples;  a  thorough  review  of  these  and  other  efforts 
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is  provided  by  Hammond  and  his  colleagues  [1980].  These  studies 
of  the  combining  of  cues  to  form  judgements  are  rather  different 
than  the  types  of  task  discussed  thus  far  in  this  paper,  in  that 
the  combination  rules  that  are  identified  do  not  necessarily 
directly  relate  to  any  explicit  model  of  the  system. 
Nevertheless,  the  whole  issue  of  cue  utilization  is  very 
important  and  is  discussed  further  later  in  this  paper. 

Analytical  Modeling 

There  are  very  few  tasks  where  empirical  modeling  is 
appropriate.  For  most  tasks,  the  input-output  relationships 
identified  would  be  very  likely  to  be  confounded  with 
characteristics  of  displaya  and  controls,  as  well  as  subjects’ 
interpretations  of  performance  criteria.  Analytical  modeling  is 
a  common  approach  to  these  types  of  task,  particularly  in  the 
manual/supervisory  control  community. 

Analytical  modeling  involves  using  available  theory  and  data 
to  formulate  assumptions  about  the  form,  structure,  and  perhaps 
parameters  of  mental  models  for  particular  tasks.  Based  on  these 
assumptions,  human  performance  (e.g.,  RMS  tracking  error)  is 
calculated  or  computed  analytically  and  compared  to  empirical 
performance  data.  A  common  practice  is  to  adjust  the  parameters 
of  the  assumed  mental  model  in  order  to  minimize  differences 
between  the  analytical  and  empirical  performance  metrics.  If  the 
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resulting  differences  are  fairly  small,  one  can  conclude  that  the 
assumed  mental  model  is  a  reasonable  approximation  for  the 
purpose  of  predicting  the  performance  metric  of  interest.  In 
contrast,  one  cannot  safely  conclude  that  one  has  identified  the 
"real"  mental  model.  Unfortunately,  this  leap,  perhaps  of  faith, 
occurs  not  infrequently. 

The  nature  of  some  domains  virtually  dictates  the  use  of 
analytical  modeling.  Neural  information  processing  is  a  good 
example  where  basic  knowledge  of  neuron  behavior  is  used  to 
synthesize  network  models.  The  overall  behaviors  of  these 
network  models  are  analytically  determined  and  compared  to 
empirical  results  of  basic  psychological  studies  [Anderson, 
1983].  The  complexity  of  the  neural  system  is  such  that  a  purely 
empirical  approach  is  simply  not  feasible. 

As  noted  earlier,  analytical  modeling  is  quite  common  in  the 
manual/supervisory  control  domain.  Because  of  the  very 
constrained  nature  of  many  manual  control  environments  (i.e.,  the 
human  must  adapt  to  the  task  in  order  to  perform  acceptably),  a 
common  assumption  is  that  humans'  mental  models  are  perfect 
relative  to  the  real  system  (e.g.,  [Kleinman,  et  al.,  1971]). 
However,  for  tasks  involving  only  monitoring  [Smallwood,  1967; 
Sheridan,  1970],  especially  when  apparent  discontinuities  occur 
in  the  state  trajectory  [Cagalayan  and  Baron,  1981],  imperfect 
models  are  often  assumed.  Imperfect  mental  models  are  also 
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assumed  for  tasks  that  involve  slowly-responding  systems  such  as 
ships  and  process  plants  [Veldhuyzen  and  Stassen,  1977],  where 
the  human  has  much  greater  discretion  in  terms  of  the  timing  and 
magnitude  of  control  actions. 

The  assumption  of  an  imperfect  mental  model  can  be 
problematic  from  an  analytical  point  of  view.  If  a  perfect 
mental  model  can  be  assumed,  one  need  only  perform  an  engineering 
analysis  of  the  system  of  interest  to  identify  the  model.  In  a 
sense,  there  is  only  one  choice.  In  contrast,  there  is  an 
infinity  of  alternative  imperfect  models,  and  justifying  the 
choice  of  any  particular  alternative  can  be  difficult.  Of 
course,  if  one's  objective  is  solely  the  prediction  of  some 
overall  performance  metric,  this  difficulty  may  be  minor. 
However,  the  fact  that  one  is  able  to  "match"  such  an  overall 
metric  does  not  imply  that  one  can  reasonably  conclude  that  the 
imperfections  assumed  in  the  analytical  model  are  identical  to 
the  actual  imperfections  inherent  in  the  human's  mental  model. 

Direct  Inquiry 

Perhaps  an  obvious  alternative  to  the  somewhat  indirect 
methods  of  identification  discussed  above  is  simply  to  ask  people 
about  their  mental  models.  Introspection,  in  a  variety  of  forms, 
was  a  common  approach  to  psychological  research  in  the  1?th 
century,  particularly  in  Europe.  However,  the  behaviorist 
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movement  of  Watson  [1914]  and  later  Skinner  [1938]  almost 
completely  divested  this  approach  cf  any  credibility  it  may  have 
had  within  experimental  psychology.  Fortunately,  the  last  two 
decades  have  produced  a  substantial  softening  of  the  strict 
behaviorist  perspective.  Nevertheless,  psychologists'  yearning 
to  be  like  physicists  still  persists  to  some  extent,  despite 
fundamental  and  irreducible  differences  between  the  two  domains 
of  study  [Rouse,  1982]. 

An  approach  to  introspection  that  has  gained  substantial 
currency  of  late  is  the  verbal  protocol,  which  is  simply  a 
transcript  of  a  human  "thinking  aloud"  as  he  or  she  performs  a 
task.  Insightful  analyses  of  verbal  protocols  have  been 
performed  for  troubleshooting  [Rasmussen  and  Jensen,  1974], 
process  control  [Bainbridge,  1979],  device  understanding 
[Williams,  et  al.,  1983],  problem  solving  in  elementary  physics 
[Gentner  and  Gentner,  1983],  and  various  game-like  tasks  [Newell 
and  Simon,  1972].  In  the  cognitive  science  domain,  there  are 
many  examples  of  verbal  protocols  serving  as  the  "data"  from 
experiments ;  see  [Gentner  and  Stevens,  1983]. 

While  there  are  strong  advocates  of  this  approach  in  the 
manual/supervisory  control  community  [Bainbridge,  1979; 
Rasmussen,  1979,  1983]  as  well  as  the  cognitive  science  community 
[Newell  and  Simon,  1972;  Ericsson  and  Simon,  1980,  1984],  there 
are  also  more  circumspect  views  [Nisbett  and  Wilson,  1977]. 
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Certainly,  what  humans  say  they  are  thinking  about  or  intend  to 
do  is  interesting  and  of  value.  However,  verbalization  of  a 
non-verbal  (e.g.,  spatial  or  pictorial)  image  may  result  in 
severe  distortions  and  biases.  Further,  verbal  protocols 
provide,  at  best,  information  about  what  humans  are  thinking 
about,  but  little  direct  information  about  how  they  are  thinking 
(i.e.,  about  the  underlying  information  processing).  Therefore, 
it  seems  prudent  to  view  verbal  protocols  as  quite  useful,  but 
far  from  conclusive.  As  a  result,  such  data  may  be  more  useful 
for  generating  hypotheses  for  subsequent  experimentation  rather 
than  as  a  primary  means  for  testing  hypotheses  (unless,  of 
course,  the  hypotheses  only  address  the  "what"  of  thinking). 

Another  approach  to  direct  identification  of  mental  models 
is  interviews  and/or  questionnaires.  In  general,  this  approach 
is  quite  different  from  verbal  protocols.  However,  in  some 
cases,  the  only  difference  between  this  approach  and  verbal 
protocols  is  the  fact  that  the  inquiry  does  not  occur  as  the  task 
is  performed.  Studies  of  air  traffic  control  by  Falzon  [1981] 
and  Whitfield  and  Jackson  [1982],  and  of  marine  navigation  by 
Hutchins  [1983],  are  of  this  type. 

In  contrast,  interviews  and/or  questionnaires  concerning 
preferences  or  judgements  are  not  necessarily  task-oriented.  In 
such  cases,  there  is  really  no  reason  to  make  inquiries  during 
task  performance.  An  excellent  example  of  this  type  of  situation 
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is  the  study  of  "mental  maps11  by  Gould  and  White  [1974],  where 
the  concern  was  with  geographical  perceptions  and  preferences. 
(Wickens  [1984,  pp.  189-192]  reviews  a  variety  of  studies  of  how 
humans'  mental  representations  of  imagined  maps  tend  to  be 
distorted. ) 

As  an  interesting  aside,  the  above  observations  on  direct 
inquiry  have  important  implications  for  the  design  of  "expert 
systems."  Succinctly,  experts  may  not  be  able  to  verbalize  their 
expertise.  Perhaps  worse,  their  verbalizations  may  reflect  what 
they  expect  is  wanted  by  the  inquirer  rather  than  how  they 
actually  perform.  An  example  of  evidence  of  this  phenomenon  is  a 
recent  study  of  process  control  operators  whose  explanations  of 
what  they  thought  they  would  (or  perhaps  should)  do  were  at 
variance  with  their  actual  behaviors  [Morris  and  Rouse,  1985; 
Knaeuper  and  Rouse,  19851. 


Summar: 


Reconsidering  all  of  the  approaches  to  identification 
discussed  in  this  section,  it  is  clear  that  each  type  of  approach 
has  substantial  advantages  for  some  types  of  task,  but  also 
important  weaknesses.  Further,  while  employing  multiple 
approaches  can  compensate  for  these  weaknesses  to  an  extent,  the 
possibility  of  totally  "capturing"  the  mental  model  is  rather 
remote.  This  is,  in  part,  due  to  the  great  likelihood  that  a 
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mental  model  does  not  exist  as  a  static  entity  having  only  a 
single  form. 

TAXONOMIES 

It  is  fairly  easy  to  accept  the  assertion  that  any 
particular  phenomenon  can  be  thought  of  in  a  variety  of  ways. 
For  example,  one  can  think  of  an  automobile  as  a  collection  of 
electromechanical  elements  that  convert  chemical  energy  of  fuel 
to  mechanical  energy  in  terms  of  motion.  In  contrast,  one  can 
view  an  automobile  as  a  sleek,  sculptured,  and  powerful  extension 
of  one's  persona.  Both  of  these  "mental  models"  involve  the  same 
physical  entity.  However,  the  verbal  protocols  produced  for 
these  two  models  of  an  automobile  would  differ  in  rather  dramatic 
ways.  This  would  be  the  case  even  if  the  two  protocols  were 
produced  by  the  same  individual. 

As  noted  earlier,  Rasmussen  [1979]  has  developed  a  taxonomy 
of  alternative  mental  models  of  systems.  His  taxonomy  moves  from 
concrete  to  abstract  perspectives  in  terms  of  five  types  of 
model:  1)  physical  form,  2)  physical  function,  3)  functional 
structure,  4)  abstract  function,  and  5)  functional  meaning  or 
purpose.  Thus,  roughly  speaking,  a  system  can  be  viewed  as  what 
it  looks  like,  how  it  functions,  or  why  it  exists.  All  of  these 
views  are  "correct"  and  of  value  for  answering  a  variety  of 
questions  about  a  system. 


Norman  [1983]  uses  the  word  "conceptualization"  to 
characterize  researchers'  models  of  humans'  mental  models*  This 
characterization  serves  to  emphasize  the  difficulty  of  studying 
mental  models  in  that  one  is  basically  searching  for 
approximations  of  approximations  of  reality  [Cohen  and  Murphy, 
1984],  a  process  that  can  be  viewed  as  akin  to  estimating  the 
variance  of  the  variance  in  statistical  modeling. 

The  conceptualizations  chosen  by  researchers  tend  to  reflect 
their  methodological  backgrounds  and  the  way  in  which  they  assume 
humans  are  likely  to  view  the  systems  of  interest*  Assumptions 
about  how  people  view  systems  are,  of  course,  also  likely  to  be 
affected  by  researchers'  backgrounds  (e.g.,  engineers  may  think 
that  operators  and  maintainers  view  systems  from  an  engineering 
perspective).  Thus,  researchers'  mental  models  affect  their 
conceptualization  of  other  humans'  mental  models;  to  avoid 
getting  sidetracked  by  this  issue,  it  is  not  pursued  further 
until  a  later  section  of  this  paper. 

A  practical  implication  of  this  phenomenon  is  that  it  is 
quite  natural  to  taxonomize  mental  models  in  terms  of 
conceptualizations.  In  reviewing  how  researchers  have  approached 
human  detection  and  diagnosis  of  system  failures,  Rasmussen  and 
Rouse  [1981]  contrast  conceptualizations  involving  differential 
equations,  functional  block  diagrams,  and  "snapshots"  of  physical 
form  as  examples  of  different  ways  that  various  researchers  view 
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similar  problems.  Beyond  differences  in  conceptualizations 
dictated  by  researchers'  natural  inclinations,  there  are 
important,  and  hopefully  more  substantial,  effects  of  differences 
in  how  mental  models  are  used. 

Young  [19833  has  suggested  a  range  of  uses  of  mental  models. 
For  example,  a  mental  model  might  be  used  as  a  way  of  describing 
a  device  independent  of  its  usage.  Another  use  of  a  mental  model 
of  a  device  might  be  to  represent  the  input-output  relationships 
associated  with  typical  uses  of  the  device.  Yet  another  use  of  a 
mental  model  of  a  device  is  as  a  means  of  understanding  an 
analogous  device  (e.g.,  a  VDU  is  like  a  typewriter). 

The  clear  Implication  of  such  usage-oriented  perspectives  is 
that  humans'  mental  models  of  a  system  (e.g.,  within  Rasmussen's 
taxonomy),  and  the  most  appropriate  conceptualizations  of  these 
models,  depend  upon  the  tasks  to  be  performed.  If  the  system  is 
used  in  multiple  ways  (e.g.,  the  automobile  example  noted 
earlier),  then  multiple  mental  models  are  likely  to  be  developed. 

Therefore,  a  taxonomy  that  is  purely  system  oriented  (i.e., 
task  independent),  will  be,  at  best,  inadequate;  a 
behavior-oriented  framework  is  also  needed.  Of  course, 
approaching  mental  models,  or  cognition  In  general,  from  a 
behavior  or  performance  point  of  view  is  the  norm  in  experimental 
psychology.  Taxonomic  efforts  in  this  discipline  tend  to  produce 
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attributes-oriented  characterizations  for  particular  tasks.  For 
example,  Wickens  [1984]  discusses  specificity  and  code  of 
representation  as  attributes  of  mental  models  in  process  control. 

From  the  foregoing  discussion,  it  is  clear  that  efforts  to 
develop  taxonomies  of  mental  models  are  heavily  influenced  by  the 
domain  being  investigated  (e.g.,  word  processing  vs.  vehicle 
control),  as  well  as  the  backgrounds  of  the  investigators  (e.g., 
psychology  vs.  engineering  vs.  computer  science).  Research  in 
a  wide  variety  of  domains  can  be  characterized  as  dealing  with 
mental  models.  Thus,  the  literature  cited  in  this  paper  includes 
several  domains:  1)  neural  information  processing,  2)  manual 
control,  3)  supervisory  control,  4)  understanding  of  devices 
(e.g.,  for  maintenance  purposes),  5)  problem  solving  in  physics, 
and  6)  making  value  judgements. 

While  all  of  ths  research  cited  in  these  domains  explicitly 
deals  with  mental  models  (or  equivalent  concepts),  these  efforts 
differ  substantially  in  terms  of  conceptualizations  chosen  and 
identification  methods  employed.  It  appears  that  these 
differences  can  bs  explained  by  distinctions  among  domains  along 
two  dimensions:  1)  nature  of  model  manipulation,  and  2)  level  of 
behavioral  discretion.  The  distinctions  among  the  various 
domains  listed  above  are  illustrated  in  terms  of  these  two 
dimensions  in  Figure  2.  (Note  that  "understanding  of  devices" 
appears  as  "system  maintenance"  and  "using  assembly 
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instructions . " ) 

The  nature  of  model  manipulation  can  range  from  implicit  to 
explicit,  where  these  terms  refer  to  whether  or  not  a  human  is 
aware  of  his  or  her  manipulation  of  a  mental  model.  As  an 
example,  one  is  likely  to  be  totally  unaware  of  manipulating 
neural  network  representations  in  associative  memory.  In 
contrast,  assembling  devices  or  solving  physics  problems  is 
likely  to  involve  explicit  manipulation  of  models. 

An  alternative  point  of  view  relative  to  this  dimension  is 
to  consider  the  terms  "implicit"  and  "explicit"  as  indicative  of 
a  dichotomy  rather  than  end  points  on  a  continuum.  The  result  is 
an  analogy  of  the  compiled  vs.  interpreted  processes  of  Newell 
and  Simon  [1972].  One  can  also  express  this  difference  in  terms 
of  systems  vs.  applications  software.  The  basic  Idea  is  that 
the  "source  code"  for  compiled  processes  or  systems  software  is 
no  longer  available  to  the  human  who,  therefore,  cannot  report  on 
how  it  operates. 

The  l9Ve^  °f  behavioral  discretion  can  range  from  none  to 
full,  where,  as  above,  these  terms  refer  to  the  extent  that  a 
human’s  behavior  is  a  matter  of  choice,  as  opposed  to  being 
dictated  by  the  task.  At  one  extreme,  phenomena  such  as  neural 
information  processing  are  unlikely  to  be  discretionary. 
However,  as  tasks  are  more  oriented  toward  decision  making  and 
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problem  solving,  opportunities  for  discretion  are  more  likely* 
Interestingly,  humans'  roles  in  many  engineering  systems  are 
tending  toward  tasks  that  involve  greater  discretion;  the  more 
task-dominated  aspects  of  system  operations  are  being 
increasingly  automated. 

While  the  relative  placement  of  domains  in  Figure  2  is  far 
from  exact,  the  distinctions  emphasized  in  this  figure  provide  a 
basis  for  explaining  methodological  differences  among  domains. 
Considering  identification  methods,  two  generalizations  seem 
reasonable. 

First,  inferential  methods  (i„e.,  empirical  assessment, 
empirical  modeling,  and  analytical  modeling)  tend  to  yield  more 
accurate  descriptions  when  there  is  little  discretion.  This  is 
because  the  nature  of  the  conceptualization  of  a  mental  model  can 
be  based  on  external  environmental  and  organizational 
constraints.  Since  the  human  has  little  discretion,  he  or  she 
can  be  assumed  to  adapt  to  these  constraints  and  the  resulting 
mental  model  will  reflect  this  adaptation. 

The  second  generalization  is  that  verbalization  methods 
(i.e.,  verbal  protocols,  interviews,  and  questionnaires)  are 
likely  to  provide  more  appropriate  descriptions  when  there  is 
explicit  manipulation.  This  is  simply  due  to  the  fact  that  the 
need  for  explicit  manipulation  may  result  in  verbalization  being 


a  "natural”  part  of  a  task.  Of  course,  it  is  also  quite  possible 
that  manipulation  may  be  explicit,  but  the  mental  model  is,  for 
example,  spatial  rather  than  verbal,  or  perhaps  in  terms  of 
subjective  images  rather  than  objective  constructs. 

If  accepted,  these  two  generalizations  have  important 
implications.  Most  obvious  is  the  conclusion  that  domains  toward 
the  upper  left  of  Figure  2  are  likely  to  present  methodological 
difficulties,  at  least  in  the  sense  that  mental  models  will  be 
elusive.  An  example  is  the  aforementioned  research  on  human 
judgement  (e.g.,  [Hammond,  1980]),  which  attempts  to  "capture" 
relationships  between  features  observed  and  decisions  made. 

The  results  of  such  analyses  indicate,  at  most,  what  Is 
taken  into  account  in  the  process  of  social  decision  making,  but 
not  how  this  information  is  processed  in  the  context  of  one  or 
more  mental  models.  The  types  of  situation  addressed  are  too 
laden  with  implicit  values  and  too  open  to  discretion  to  allow 
mental  models  to  be  "captured"  to  the  extent  that  they  can  be, 
for  example,  for  device  understanding.  Studies  of  human  judgment 
in  the  area  of  personal  relations  [Harvard,  1980]  and  personal 
geographical  preferences  [Gould  and  White,  1974]  are  good 
examples  of  this  limitation. 

Expanding  upon  the  above  notion,  an  overall  implication  of 
the  generalizations  drawn  from  Figure  2  is  that  the  possible 
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level  of  specificity  of  conceptualizations  of  mental  models,  and 
perhaps  even  the  form  of  conceptualizations,  are  limited  by  the 
location  of  a  task  domain  along  the  nature  of  manipulation/level 
of  discretion  dimensions.  In  fact,  it  seems  reasonable  to 
conjecture  that  these  limits  may  be  fundamental.  Elaboration  of 
this  conjecture  is,  however,  delayed  until  a  later  section  of 
this  paper. 

SALIENT  ISSUES 

From  the  discussion  thus  far,  it  is  clear  that  there  are  a 
plethora  of  issues  surrounding  the  topic  of  mental  models.  Many 
of  these  are  relatively  minor,  involving  terminology  and  inherent 
differences  among  domains.  A  few  issues,  however,  appear 
repeatedly  in  the  literature  and  are  dominant  in  many  of  the 
domains  discussed  in  this  paper. 

This  section,  as  well  as  the  following  section,  explore  the 
nature  of  these  issues.  The  discussion  proceeds  in  the  following 
sequence : 


1.  Accessibility  -  To  what  extent  is  it  possible  to  "capture" 
individuals'  mental  models? 


2.  Forms  of  representation  -  What  do  mental  models  look  like 
(e.g.,  spatial  vs.  verbal)? 


3. 


Context  of  representation  -  To  what  extent  can  mental  models 
be  general  rather  than  totally  context-dependent? 
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4®  Nature  of  expertise  -  How  do  the  mental  models  of  novices  and 
experts  differ? 

5.  Cue  utilization  -  How  are  mental  models  affected  by  the  cues 
one  employs,  either  by  choice  or  due  to  availability? 

6.  Instruction  -  How  can  and  should  training  affect  individuals' 
mental  models? 

The  rationale  underlying  the  ordering  of  these  topics  is  to 
consider  first  the  inherent  nature  of  mental  models,  particularly 
as  affected  by  context,  expertise,  and  available  cues,  and  then 
to  focus  on  approaches  to  fostering  the  development  of 
appropriate  mental  models. 

Accessibility 

As  might  be  surmised  from  the  foregoing  discussion,  the 
accessibility  of  mental  models  is  a  recurrent  and  important 
issue.  While  the  considerations  outlined  earlier  need  not  be 
repeated,  it  is  of  value  to  note  a  few  examples  where 
accessibility  appears  limited  in  the  sense  that  researchers' 
abilities  to  "capture"  mental  models  are  constrained  by  humans' 
lack  of  abilities  to  verbalize  their  models.  Van  Heusden  [1980] 
found  that  subjects  had  difficulty  verbalizing  how  they  predictsa 
future  states  qf  time  series.  Whitfield  and  Jackson  [1982] 
reported  that  air  traffic  controllers  had  difficulty  verbalizing 
their  "picture"  of  the  state  of  the  system.  Wickens  [1984]  notes 
that  models  for  control  are  less  verbalizable  than  models  for 


detection  and  diagnosis.  As  noted  earlier,  Morris  and  Rouse 
[1985]  and  Knaeuper  and  Rouse  [1985]  found  that  subjects'  answers 
regarding  what  they  would  (or  perhaps  should)  do  were  different 
from  what  they  actually  did.  Therefore,  while  the  intent  is  not 
to  belabor  the  point,  an  important  issue  concerns  when 
verbalization  is  possible,  reliable,  and  valid.  (The  previous 
discussion  surrounding  Figure  2  suggests  how  this  issue  might  be 
viewed) . 

Forms  of  Representation 

The  accessibility  of  mental  models }  as  well  as  their  use  in 
general,  depends  on  their  forms  of  representation.  This  issue 
concerns  how  mental  models  are  encoded  and  perhaps  evolve.  While 
neural  information  processing  approaches  to  this  issue  are 
emerging  [Anderson,  198.3],  the  potential  of  such  fine-grained 
descriptions  appears,  at  least  at  this  point  in  time,,  to  be 
limited  to  providing  explanations  of  very  elementary 
psychological  phenomena  rather  than  behavior  jn  realistically 
complex  taek3. 

One  important  distinction  relative  to  form  is  spatial  vs. 
verbal.  Considering  humans'  exquisite  pattern  recognition 
abilities,  it  is  likely  that  the  human  information  processing 
system  is  particularly  adept  at  processing  spatially-oriented 
information  and,  hence,  may  tend  to  store  information  in  that 


PAGE  27 


manner.  Therefore,  it  seems  reasonable  to  suggest  that  mental 
models  are  frequently  pictorial  or  image-like  rather  than 
symbolic  in  a  list-processing  sense.  This  obviously  presents 
difficulties  when  humans  are  asked  to  verbalize  their  models 
(e.g.,  the  air  traffic  controllers  of  Whitfield  and  Jackson 
[1982]). 

Even  when  verbal  representations  are  likely  (or  at  least 
useful),  the  vocabulary  or  '’ontology”  of  such  descriptions  can  be 
an  important  factor  in  the  effectiveness  of  these  representations 
for  problem  solving  [Greeno,  1 983 3  -  An  excellent  example  is  that 
reported  by  Falzon  [1982]  where  air  traffic  controllers  thought 
of  their  task  in  terms  of  aircraft  "separations"  rather  than 
"positions.  •' 

Another  important  distinction  relative  to  form  is 
representational  vs.  abstract.  Rasmussen's  taxonomy  of  mental 
models  illustrates  how  any  particular  system  can  be  described  at 
various  points  along  this  dimension  [Rasmussen,  1979].  Larkin 
[1983]  distinguishes  expert  from  novice  solvers  of  physics 
problem  in  terms  of  abstract  vs.  representational  mental  models. 

Context  of  Representation 

A  related  issue  concerns  the  context  of  representation, 
rather  than  the  form,  and  whether  it  is  general  or  specific 
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(e.g.,  general  principles  of  physics  or'  specific  heuristics  for 
troubleshooting  a  particular  device).  In  reviewing  the  available 
evidence  for  process  control,  Wickens  [1984]  concludes  that 
mental  models  tend  to  be  specific.  However,  if  specific 
representations  are  predominant,  it  is  difficult  to  account  for 
the  richness  of  human  problem  solving  behavior  (i.e.,  abilities 
to  solve  novel  problems).  Explanations  of  this  richess  have 
included  learning  via  metaphors  [Carroll  and  Thomas,  1982], 
analogical  problem  solving  [Steinberg,  1977;  Gentner  and 
Gentner,  1983;  Silverman,  1983],  and  use  of  multiple  models 
[Rasmussen,  1983]* 

While  the  issue  of  general  vs.  specific  knowledge  is 
certainly  not  new  (e.g.,  [Peirce,  1877]),  it  is  far  from 
resolved.  Part  of  the  difficulty  is  inherent  in  the  topic. 
Tasks  and  behavior  are  always  specific.  Hence,  "general" 
phenomena  are  not  observable.  let,  such  constructs  seem  to  be 
necessary  to  explain,  for  example,  human  behavior  in  unfamiliar 
situations  [Glaser,  1984].  Given  the  fact  that  much  of  what  is 
routine  is  increasingly  being  automated,  leaving  humans  to  deal 
with  the  non-routine,  a  recurring  theme  is  training  of  humans  to 
have  general  skills  to  deal  with  a  wider  variety  and  less 
familiar  tasks.  As  might  be  expected,  therefore,  the  general  vs. 
specific  issue  is  likely  to  continue  to  receive  attention. 


At  least  a  portion  of  the  general  vs.  specifc  debate  has 
focused  on  the  nature  of  expertise.  The  question  of  concern, 
within  the  context  of  this  paper,  is  how  experts*  mental  models 
differ  from  those  of  novices.  Intuitively,  one  might  think  that 
experts  simply  know  more  than  novices  (i.e.,  have  more  elaborate 
and  accurate  mental  models).  However,  experts'  mental  models  are 
not  just  more  elaborate  or  accurate;  evidence  suggests  that  they 
are  fundamentally  different  from  novices'  models  [Chi  and  Glaser, 
1984;  Glaser,  1984;  Greeno  and  Simon,  1984]. 

Wisner  and  Carey  [1983]  have  concluded  that  the 
"novice-expert  shift"  involves  a  conceptual  change,  rather  than 
just  refinement  of  the  novice's  perspective.  As  noted  earlier, 
Larkin  [1983]  discusses  this  shift  as  a  movement  from 
representational  to  abstract  models.  Chase  and  Simon  [1973],  as 
well  as  Dreyfus  and  Dreyfus  [1979],  describe  expertise  in  terms 
of  highly-developed  repertoires  of  pattern-oriented 
representations.  If  one  accepts  the  conclusion  that  experts  tend 
to  have  conceptually  abstract,  pattern-oriented  mental  models, 
then  one  must  simultaneously  question  the  accessibility  of  these 
models  via  verbalization  methods.  This  has,  of  course,  important 
implications  for  developers  of  "expert  systems." 

An  interesting  phenomenon  related  to  expertise  is  the  fact 
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that  the  shift  away  from  novice  does  not  necessarily  imply  that 
all  naive  notions  are  discarded.  DiSessa  [1982]  and  McCloskey 
[1983]  found  that  naive,  "pre-Newtonian"  theories  of  motion  were 
retained  by  students  even  after  instruction  in  "correct" 
theories.  Similarly,  Clement  [1983]  found  that  the  naive  idea  of 
"motion  implies  force"  was  retained  even  after  instruction  that 
indicated  otherwise.  Thus,  individuals  who  know  what  is 
"correct"  may  also  retain  ideas  that  are  "wrong,"  perhaps  because 
their  real-world  (as  opposed  to  instructional)  experiences  tend 
to  be  such  that  inconsistencies  do  not  occur.  In  other  words, 
mental  models  may  include  a  bit  of  "baggage"  remaining  from 
earlier  experiences  that  humans  find  no  need  to  question  or 
discard,  even  though  this  baggage  may  create  difficulties  when 
novel  situations  are  encountered. 

An  alternative  interpretation  of  the  above  results  is  that 
the  subjects  studied  were  not  "experts"  in  the  full  sense  of  the 
word;  otherwise,  their  naive  notions  would  have  been  dispelled. 
While  this  position  is  reasonable,  it  runs  the  risk  of  investing 
in  experts  the  non-human  characteristic  of  always  being  correct. 
Alternatively,  one  can  define  expertise  in  relative  terms.  From 
this  perspective,  the  results  cited  above  are  perhaps  suggestive 
of  the  inherent  limitations  of  expert  opinion. 
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Cue  Utilization 


An  issue  that  is  often  overlooked  in  discussion  of  mental, 
models  is  cue  utilization.  In  order  to  predict  future  system 
states  or  explain  the  current  state,  two  things  r.re  ne3ded:  1  ) 
one  has  to  know  what  the  current  state  is,  and  2)  one  has  to  have 
some  mechanism  that  emulates  the  proceas  whereby  the  state 
evolves.  The  human’s  internalization  of  this  mechanism  is 
usually  thought  of  as  the  mental  model;  however,  the  development 
and  use  of  this  mechanism  cannot  be  divorced  from  the  human' 3 
abilities  to  extract  from  the  environment  the  cues  necessary  to 
form  the  state  estimates  upon  which  this  mechanism  operates. 

An  excellent  example  of  possible  confounding  of  cue 
utilization  and  mental  models  can  be  found  in  various  studies  of 
humans'  abilities  to  predict  future  system  states.  Independent 
studies  by  Rouse  [1977],  van  Bussel  [1980],  and  van  Heusden 
[1980]  have  concluded,  via  empirical  modeling  methods,  that 
humans'  models  reflect  inappropriate  weightings  of  past  system 
states.  All  three  of  these  efforts  assumed  that  past  states  were 
accurately  observed,  or  at  most  were  subject  to  zero-mean 
Gaussian  observation  noise. 

However,  despite  these  researchers'  serious  efforts  to  avoid 
it,  subjects  may  have  produced  consistently  biased  or  distorted 
state  estimates  which  led  them  to  develop  what  appeared  to  be 


inappropriate  mental  models.  For  example,  subjects  may  have 
looked  for  spatial  patterns  such  as  number  of  reversals  or 
repeated  subpatterns  in  the  displayed  time  series  rather  using 
the  "state”  as  the  investigators  had  intended.  If  this  was  the 
case,  it  may  have  been  that  the  mental  models  developed  by 
subjects  were  "optimal"  (i.e.,  the  best  fit)  for  those  cues.  In 
other  words,  it  may  have  been  that  their  cue  utilization  dictated 
the  limits  to  the  accuracy  of  their  models. 

This  phenomenon  has  implications  for  explaining  the  impact 
of  predictor  displays.  A  predictor  display  explicitly  depicts, 
via  a  model  of  the  system,  the  future  states  of  the  system  and 
has  been  shown  to  result  in  improved  system  performance  [Sheridan 
and  Ferrell,  1974,  pp«  268-273).  One  explanation  for  this 
improvement  is  that  humans’  mental  models  of  the  systems  involved 
were  other  than  perfect.  Alternatively,  as  argued  above,  it 
could  be  that  they  simply  tended  to  have  difficulty  estimating 
the  higher-order  3tate  variables  (e.g.,  acceleration  and  its 
derivatives ) , 

A  study  by  Johannsen  and  Govindaraj  [1980]  supports  the 
latter  hypothesis.  They  used  a  manual  control  model  to  assess 
the  effects  of  a  predictor  display,  which  they  represented  solely 
in  terms  of  improved  cue  utilization.  Experimental  data 
supported  their  formulation,  although  their  study  was  designed 
for  purposes  other  than  providing  a  definitive  test  of  the  cue 
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utilization  vs*  imperfect  mental  model  issue. 

Increasing  levels  of  automation  in  engineering  systems  have 
led  to  a  variety  of  studies  of  the  impact  on  human  performance  of 
manually  controlling  vs.  monitoring  of  automatic  controls  in 
tasks  such  as  failure  detection.  Kessel  and  Wickens  [1982]  found 
that  subjects  trained  in  failure  detection  while  manually 
controlling  subsequently  produced  better  failure  detection 
performance  when  monitoring  an  automatically  controlled  system. 
They  concluded  that  training  that  included  manual  control  leads 
to  improved  cue  utilization.  Ephrath  and  Young  [1981]  reach  what 
at  first  glance  appears  to  be  almost  the  opposite  conclusion  but, 
upon  closer  inspection,  mainly  serves  to  illustrate  the 
subtleties  of  the  issue.  (For  example,  the  value  of  information 
is  related  to  the  human  information  processing  resources  required 
to  utilize  the  information.)  In  a  rather  different  study,  but 
still  within  the  manual  control  domain,  Cohen  and  Ferrell  [1967] 
found  that  subjects'  abilities  to  estimate  "readiness"  of  the 
driver  to  perform  difficult  maneuvers  with  an  automobile  were  no 
different  if  they  were  to  perform  the  maneuver  themselves  or  they 
were  simply  observing  another  driver  (i.e.,  manual  involvement 
did  not  enhance  performance). 

The  above  studies  on  prediction,  predictor  displays,  and 
manual  control  mainly  serve  to  emphasize  the  importance  of  cue 
utilization  in  development  and  use  of  mental  models.  Succinctly, 
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one's  conceptualization  of  how  something  works  is  highly- 
influenced  by  what  observations  one  chooses  to  make.  Therefore, 
when  attempting  to  identify  the  cause  of  suboptimal  performance 
by  humans,  one  should  try  to  avoid  confounding  information 
processing  limits  (e.g.,  memory)  and  inappropriate  or  inadequate 
cue  utilization.  In  some  situations,  these  two  types  of 
limitation  seem  to  have  demonstrably  different  effects  [Baron  and 
Berliner,  1977].  However,  in  general  it  appears  that 
insufficient  attention  has  been  devoted  to  this  issue. 

An  interesting  aspect  of  cue  utilization  is  the  extent  to 

i 

which  it  differs  for  novices  and  experts.  In  general,  experts 
are  not  found  to  be  unduly  influenced  by  superficial  cues  [Chi 
and  Glaser,  1984].  For  example,  in  a  study  of  the  use  of 
research  literature,  Morehead  and  Rouse  [1985]  found  that  faculty 
members  were  much  more  definitive  than  Ph.D.  students  in 
specifying  attributes  of  information  that  they  did  not  want 
retrieved.  However,  there  are  situations  where  novices  perform 
relatively  better  because  they  utilize  more  concrete,  detailed 
representations  [Adelson,  1984].  Nevertheless,  available 
evidence  indicates  that  an  important  attribute  of  expertise  is 
the  ability  to  select  the  most  useful  features  of  problems. 

A  Central  Issue 


To  the  extent  that  it  is  reasonable  to 


characterize  any 
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singls  issue  as  the  central  issue,  that  issue  has  to  be 
instruction  or  training.  For  any  particular  task,  job,  or 
profession,  what  mental  models  should  humans  have  and  how  shopld 
these  models  be  imparted?  This  question  is  of  sufficient 
theoretical  and  practical  importance  to  warrant  a  much  more 
detailed  treatment  than  accorded  to  the  other  salient  issues 
considered  in  this  section. 

INSTRUCTIONAL  ISSUES 

The  purpose  of  instruction  is  to  provide  the  learner  with 
necessary  knowledge  and  skills,  as  well  as  improve  confidence, 
attitude,  etc.  For  instruction  related  to  any  given  system,  a 
subset  of  the  necessary  knowledge  and  skills  relates  to  the 
ability  to  describe  purpose  and  form,  explain  functions  and 
observed  states,  and  predict  future  states.  Therefore,  one  of 
the  purposes  of  instruction  is  to  provide  necessary  mental 
models . 

While  this  may  seem,  at  least  initially,  straightforward,  it 
is  a  very  difficult  issue.  The  basic  questions  are:  For  a  given 
system,  what  do  the  humans  involved  with  that  system  need  to  be 
able  to  do,  and  what  knowledge  is  necessary  for  them  to  develop 
and  maintain  this  repertoire  of  skills?  An  important  related 
question  is:  What  is  the  most  appropriate  form  for  this 
knowledge? 
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Within  this  section,  these  questions  are  considered  in  terms 
of  the  types  of  .  nowledge  included  within  the  proposed  definition 
of  mental  models.  For  the  most  part,  this  discussion  emphasizes 
the  impacts  of  particular  types  of  knowledge  rather  than  the  more 
global  concepts  of  mental  models.  This  level  of  specificity 
serves  to  emphasize  the  potential  utility  of  many  of  the  lesults 
cited . * 

Knowledge  of  Theories  and  Principles 

When  considering  the  questions  noted  above,  a  fairly  common 
assertion  is  that  humans  (particularly  operators  and  maintainers) 
need  to  understand  thoroughly  the  fundamental  principles  upon 
which  the  design  and  operation  of  the  system  of  interest  is 
based.  The  '’principles”  of  concern  usually  include  fundamentals 
of  thermodynamics,  heat  transfer,  fluid  mechanics,  solid 
mechanics,  dynamics,  electricity,  and  perhaps  mathematics.  Many 
technical  training  programs  place  heavy  emphasis  on  these  types 
cf  principle. 

Unfortunately,  there  i3  little  if  any  evidence  that  this 
emphasis  results  in  better  and  more  useful  mental  models.  In  tho 

*  The  need  for  this  level  of  specificity  also  serves  to  highlight 
the  fact  that  expressing  results  solely  in  terms  of  global  and 
somewhat  vague  concepts  tends  to  dissipate  any  impact  these 
results  might  potentially  have. 


domain  of  process  control,  a  variety  of  independent  studies  have 
shown  that  explicit  training  in  knowledge  of  theories, 
fundamentals,  or  principles  did  not  enhance  performance,  and 
sometimes  actually  degraded  performance  [Crossman  and  Cooke, 
1962;  Kragt  and  Landeweerd,  1974s  Brigham  and  Laios,  1975; 
Shepherd,  et  al.,  1977;  Morris  and  Rouse,  1985].  It  has  also 
been  found  that  scores  on  tests  of  fundamental  understanding  did 
not  correlate  significantly  with  process  control  performance 
[Surgenor  and  McGeachy,  1983;  Morris  and  Rouse,  1985]. 

Similar  results  have  been  found  in  the  domain  of  electronics 
troubleshooting.  Schorgmayer  and  Swanson  [1975]  determined  that 
an  account  of  system  functioning  did  not  enhance  performance 
relative  to  procedural  assistance.  Williams  and  Whitmore  [1959] 
found  that  knowledge  of  theory  was  greatest  and  troubleshooting 
performance  poorest  immediately  following  training;  the  opposite 
conclusions  were  reached  when  the  same  subjects  were  tested  three 
years  later.  Foley  [1977]  reviewed  seven  studies  of 
troubleshooting,  including  that  of  Williams  and  Whitmore,  and 
concluded  that  performance  on  tests  of  theory  and  job  knowledge 
did  not  correlate  with  actual  job  performance. 

Results  in  the  domain  of  mathematical  problem  solving  are 
also  similar.  Two  studies  compared  training  that  emphasized 
general  understanding  of  mathematical  principles  to  training  that 
stressed  calculations!  techniques  [Mayer  and  Greeno,  1972; 
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Guidance  and  Cueing 

Guidance  in  the  use  of  knowledge  can  occur  in  several  ways. 
Many  of  the  studies  noted  above  provided  trainees  with  explicit 
procedures  for  performing  their  tasks.  In  some  cases,  the 
comparison  was  procedures  vs.  principles;  in  other  cases, 
training  via  procedures  served  as  more  of  a  control  group.  In 
general,  procedures  tended  to  be  at  least  as  useful  as 
principles,  and  at  least  as  useful  as  having  both  procedures  and 
principles. 

•  \ 

Procedures  represent  an  extreme  form  of  converting  general 
principles  into  operationally-useful  guidance.  A  less  extreme 
form  cf  guidance  involves  simply  Informing  trainees  of  how  and 
when  the  knowledge  gained  during  training  should  be  used,  without 
telling  them  exactly  what  they  should  do.  A  variety  of  studies 
in  problem  solving  [Reed,  et  al.r  1974;  Weisberg,  et  al.,  1978], 
word  puzzles  [Perfetto,  et  al.,  1983],  and  mathematics  [Mayer,  et 
al.,  1977]  have  considered  the  effect  of  this  type  of  "cueing" 
and  found  it  to  be  necessary  if  clues,  analogies,  and  general 
principles  are  to  be  transferred  successfully  to  task  performance 
subsequent  to  training. 

It  is  not  always  possible  for  guidance  to  be  explicit.  If 
systems  are  very  complex  and/ or  completely  unanticipated 
situations  may  arise,  it  is  likely  to  be  impossible  to  synthesize 


procedures  that  can  be  validated  in  the  sense  of  assuring 
success.  Similarly,  it  may  be  impossible  to  inform  trainees  of 
how  and  when  knowledge  will  be  applicable  (i.e.,  ” cueing”  may  not 
be  viable).  Nevertheless,  one  hopes  that  the  knowledge  gained 
during  training  will  be  called  upon  when  unusual  situations 
arise. 

One  approach  to  enhancing  this  possibility  is  to  provide 
training  in  a  variety  of  contexts  (e.g.,  for  more  than  one 
system,  one  or  more  of  which  may  be  unfamiliar).  The  use  of 
unfamiliar  contexts  can  {,force”  trainees  to  utilize  general 
principles  such  as  analogies  because  that  may  be  the  only  way  in 
which  they  can  succeed.  Rouse  and  Hunt  [1984]  have  investigated 
various  aspects  of  this  concept  as  applied  to  troubleshooting 
training.  While  they  found  that  the  use  of  unfamiliar  contexts 
is  somewhat  more  subtle  and  complicated  than  originally 
anticipated,  the  concept  was  sufficiently  viable  and  useful  to 
become  an  important  element  in  training  programs  in  the  aviation 
and  marine  domains  [Rouse,  1982-831.  Brooko  and  his  colleagues 
[1980]  have  also  investigated  a  variation  of  this  concept  and 
found  that  training  in  multiple  contexts  improved  transfer  of 
problem  solving  skills  to  new  contexts. 

These  results  serve  to  emphasize  the  possibility  that  human 
performance  within  a  particular  system  context  may  be 
significantly  affected  by  their  knowledge  of  other  contexts. 
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Thus,  not  only  are  tasks  within  a  particular  system  likely  to  be 
addressed  via  multiple  mental  models  of  that  system,  but  task 
performance  may  also  be  influenced  by  mental  models  of  other 
systems  and  classes  of  systems.  This  leads  to  the  issue  of  prior 
knowledge. 

Effects  of  Prior  Knowledge 

With  the  possible  exception  of  very  young  children, 
instruction  never  involves  the  filling  of  a  tabula  rasa. 
Trainees  always  approach  an  Instructional  experience  with  prior 
knowledge  and  skills.  In  particular,  trainees  always  have  a 
variety  of  a  priori  mental  models  which  provide  both 
opportunities  and  difficulties  from  an  instructional  point  of 
view. 


The  availability  of  prior  knowledge  presents  an  opportunity 
in  that  it  can  serve  as  a  basis  for  gaining  new  knowledge.  In 
fact,  it  can  be  argued  that  prior  knowledge  will  almost  certainly 
affect  learning  [Glaser,  19843.  For  example,  in  the  domain  of 
human-computer  interaction,  Carroll  and  Thomas  [1982]  argue  that 
new  "cognitive  structures"  are  developed  by  using  metaphors  to 
existing  cognitive  structures.  Norman  and  his  colleagues  [1976] 
offer  a  similar  assertion  with  regard  to  the  design  of 
instructional  programs.  Rasmussen  [1979,  1983]  discusses 
implications  of  alternative  mental  models  for  display  design  and 
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suggests  that  analogies  offer  an  important  mechanism  for  matching 
displays  to  humans*  models.  With  regard  to  analogies,  Gentner 
and  Gentner  [1983]  found  that  the  usefulness  of  analogies  in 
solving  electricity  problems  was  greatest  when  people  used  their 
own  a  priori  analogies  rather  than  using  those  that  they  had  only 
recently  learned  as  part  of  the  instructions  associated  with  the 
equipment. 

While  existing  "cognitive  structures"  offer  a  foundation  on 
which  to  build,  they  also  can  be  an  impediment.  Prior  knowledge 
that  is  incorrect  will  not  necessarily  be  discarded  once  the 
correct  knowledge  is  provided.  Instead,  an  amalgam  of  the 
correct  and  incorrect  may  be  retained,  especially  if  the 
incorrect  aspects  are  such  that  everyday  life  experiences  are 
unlikely  to  yield  any  inconsistencies. 


This  phenomenon  has  emerged  several  times  in  studies  of 
physics  problem  solving.  As  discussed  earlier,  DiSessa  [1982] 
and  McCloskey  [1983]  both  found  that  students'  naive, 
"pre-Newtonian"  views  of  motion  persisted  even  after 
college-level  instruction  had  provided  them  with  more  appropriate 
formulations.  Similarly,  Clement  [1983]  found  that  the  "motion 
implies  force"  misconception  was  retained  after  college-level 
instruction  had  provided  the  appropriate  conceptualization.  The 
implication  of  these  findings  is  that  instruction  must  remediate 
a  priori  misconceptions  as  well  as  provide  correct  knowledge. 
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Summary 


SsBatar.-zj.ng  the  evidence  presented  in  this  section  on 
inar  uc  tonal  issues,  the  following  assertions  seem  reasonable*: 


.  Knowledge  of  theories,  fundamentals,  and  principles  does  not 
necessarily  enhance  task  performance!  measures  of  the  extent 
of  such  knowledge  are  not  good  predictors  of  task 

performance. 


2.  The  operational  utility  of  this  type  of  knowledge  is  highly 
dependent  on  the  form  in  which  it  is  presented  and  the 
guidance  in  its  use  that  is  provided. 


3.  Guidance  in  the  use  of  knowledge  can  be  explicit  in  terms  of 
procedures  and  cueing,  or  implicit  by  providing  a  range  of 
training  experiences  that  foster  or  require  the  use  of 
knowledge. 


4.  A  priori  knowledge  can  serve  as  a  powerful  basis  for  gaining 
new  knowledge  or,  if  incorrect,  an  impediment  to  gaining 
correct  knowledge;  both  cases  argue  for  consideration  of  a 
priori  knowledge  in  designing  instructional  programs. 


From  the  perspective  of  mental  models,  the  above  assertions  imply 
that  the  form  of  knowledge,  guidance  in  use  of  knowledge,  and 
prior  knowledge  all  interact  to  affect  the  development  and  use  of 
mental  models. 


^Morris  and  Rousb  (1985),  in  a  recent  comprehensive  review  of 
empirical  research  on  human  performance  in  troubleshooting  tasks, 
present  considerable  evidence  for  a  similar  set  of  assertions 
relative  to  training  for  troubleshooting  tasks. 
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the  study  of  mental  models  because,  in  effect,  such  studies 
amount,  to  one  or  more  humans  developing  models  of  other  humans' 
models  of  the  extei*nal  world.  This  dilemma  is  fundamental  in 
that  it  canxiot  be  resolved.  However,  the  effects  of  this  problem 
can  perhaps  be  lessened  if  researchers  are  aware  of  the  biases 
that  they  bring  to  a  study,  and  that  these  biases  may  not  be 
indicative  of  the  tendencies  of  the  population  of  subjects  being 
studied.  Therefore,  for  example,  it  is  important  for  scientists 
and  engineers  to  avoid  the  presumption  that  operators, 
maintainors,  and  .managers  approach  their  systems  from  a 
scientific  or  engineer iug  perspective. 

Beyond  the  limits  imposed  by  .investigators'  biases,  there 
are  difficulties  that  preclude  uncovering  the  "truth."  Several  of 
these  difficulties  are  discussed,  or  at  least  alluded  to,  in 
earlier  sections  of  this  paper.  The  discussion  of  identification 
methods  considered  several  important  limitations.  It  was  noted 
that  empirical  approaches  are  limited  by  the  fact  that  behavioral 
effects  of  access  and  manipulation  of  mental  models  may  possibly 
be  confounded  with  perception  and  response  execution.  Analytical 
approaches  that  consider  the  possibility  of  other  than  perfect 
mental  models  must  choose  among  an  infinity  of  alternative 
imperfect  models. 

In  an  attempt  to  generalize  across  domains,  it  was  suggested 
that  the  specificity  and  perhaps  the  form  of  conceptualizations 


of  mental  models  are  limited  by  the  location  of  a  domain  along 
two  dimensions!  1)  nature  of  model  manipulation,  ranging  from 
implicit  to  explicit,  and  2)  level  of  behavioral  discretion, 
ranging  from  none  to  full.  This  two-dimensional  characterization 
of  differences  among  domains  appears  to  have  clear  implications 
for  the  potential  usefulness  of  alternative  identification 
methods.  Namely,  inferential  methods  seem  to  work  best  when 
there  is  little  behavioral  discretion,  while  verbalization 
methods  appear  to  be  most  successful  when  explicit  model 
manipulation  is  inherent  to  the  task  of  interest. 

If  the  above  limitations  are,  in  fact,  fundamental,  then  the 
search  for  mental  models  will  never  comletely  eliminate 
uncertainty;  the  black  box  will  never  be  completely  transparent. 
This  type  of  problem  has  been  addressed  by  particle  physicists, 
who  ultimately  accepted  this  inherent  limitation  in  terms  of 
Heisenberg's  uncertainty  principle  [Heisenberg,  1958;  Zukav, 
1979].  The  basic  idea  is  that  one  cannot  measure  perfectly  both 
the  position  and  momentum  (the  product  of  mass  and  velocity)  of  a 
particle,  because  the  process  of  measuring  position  produces 
uncertainty  in  momentum  and  vice  versa.  Heisenberg  [1958] 
generalizes  this  notion  by  stating,  "What  we  observe  is  not 
nature  itself  but  nature  exposed  to  our  method  of  questioning." 

The  general  perspective  provided  by  this  statement,  as  well 
as  the  specifics  of  the  uncertainty  principle,  appear  to  be  quite 
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relevant  to  research  on  mental  models.  Much  of  the  literature 
implies  that  mental  models  are  static,  unitary  entities  that  can 
be  identified  if  appropriate  methods  are  employed.  However,  as 
Norman  [1983]  notes,  this  view  is  much  too  simplistic.  Available 
evidence  suggests  that  mental  models  are  more  likely  to  be 
dynamic  entities  that  can  have  a  multiplicity  of  forms. 

If,  at  least  for  the  sake  of  argument,  one  asserts  that 
mental  models  are  analogous  to  physicists'  elementary  particles 
which  are  dynamic  entities  that  can  be  in  multiple  states,  then 
it  is  quite  straightforward  to  map  the  physicists'  uncertainty 
principle  to  an  analagous  principle  for  mental  models.  The 
position  of  a  particle  is  analogous  to  the  current  state  of  a 
mental  model  (i.e.,  what  it  is  now)  and  the  velocity  (or 
momentum)  of  a  particle  is  analagous  to  the  changes  occurring  in 
a  mental  model  (i.e.,  what  it  is  becoming). 

Uncertainty  is  fundamental  in  the  following  ways.  In  order 
to  measure  perfectly  what  a  mental  model  is  now,  one  inevitably 
intrudes  on  what  the  model  is  becoming.  Less  intrusive 
measurement  methods  reduce  the  effects  on  future  model  states, 
but  increase  the  uncertainty  about  the  current  state.  Similarly, 
if  one  attempts  to  measure  perfectly  what  a  model  is  becoming,  in 
attempting  to  measure  these  changes,  one  introduces  uncertainty 
about  the  instantaneous  state  of  the  model  (i.e.,  what  it  is  now) 
relative  to  which  these  changes  are  being  measured. 
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Heisenberg’s  principle  specifies  that  the  product  of  the 
uncertainties  in  position  and  momentum  is  constant  (i.e., 
Heisenberg’s  constant!).  The  psychological  analog  of  this 
constant  is  not  apparent.  In  fact,  it  seems  reasonable  to 
conjecture  that  the  magnitude  of  this  constant  might  be  domain 
dependent  in  the  sense  that  the  dimensions  in  Figure  2  may  affect 
the  level  of  inherent  uncertainty.  Despite  the  intuitive  appeal 
of  such  a  formulation,  it  must  be  remembered,  however,  that  It  is 
totally  a  conjecture. 

\ 

This  raises  the  question  of  how  this  line  of  reasoning  might 
move  beyond  pure  conjecture.  Certainly,  more  thought  is  needed 
and  a  mathematical/logical  formulation  might  be  possible.  While 
progress  might  be  made  in  this  way,  it  is  also  possible  that  a 
limit  such  as  that  of  Godel  may  be  reached,  where  "truth"  cannot 
be  proven  and  must  simply  be  accepted  [Godel,  1962;  Guillen, 
1983].  Obviously,  the  possibility  of  such  ”meta"  limits  is  yet 
another  conjecture  at  this  point  in  time. 

This  section  has  outlined  several  fundamental  limits  in  the 
search  for  mental  models,  as  well  as  several  conjectures 
regarding  limits  to  ’’knowing  what  can  be  known.”  The  intent  of 
this  discussion  was  to  illustrate  why  pursuit  of  "truth"  may  be 
inherently  elusive,  particularly  when  studying  mental  models. 
Given  these  limits,  dogged  pursuit  of  "truth"  is  unreasonable. 
Instead,  the  emphasis  should  be  on  the  utility  of  research  on 
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mental  models  for  system  design,  instruction,  etc*  This 
pragmatic  view  of  science  is  hardly  new  [Peirce,  1878;  James, 
1907];  however,  it  often  seems  to  be  forgotten. 

CONCLUSIONS 

This  paper  has  explored  a  wide  range  of  issues  associated 
with  research  on  mental  models.  At  this  point  in  time,  this  area 
of  study  is  rife  with  terminological  inconsistencies  and  a 
preponderance  of  conjectures  rather  than  data.  This  situation 
is,  to  a  great  extent,  due  to  the  fact  that  a  variety  of 
subdisciplines  have  adopted  the  concept  of  mental  models  and 
proceeded  to  develop  their  own  terminology  and  methodology, 
independent  of  past  or  current  work  in  this  area  In  ether 
subdisciplines . 

Nowhere  is  this  situation  more  evident  than  in  the  important 
matter  of  definitions.  In  many  cases,  the  phrase  "mental  models" 
appears  to  be  simply  a  substitute  for  "knowledge"  in  general. 
Such  a  substitution  is  not  particularly  useful.  This  paper  has 
suggested  a  more  concise  working  definition,  based  on  a 
functional  perspective:  mental  models  are  the  mechanisms  whereby 
humans  generate  descriptions  of  system  purpose  and  form, 
explanations  of  system  functioning  and  observed  systems  states, 
and  predictions  of  future  system  states.  Much  of  the  discussion 
in  this  paper  is  premised  on  this  definition. 


A  portion  of  thin  discussion  has  focused,  on  limits  in 
identifying  or  capturing  mental  models.  Some  of  the  difficulties 
in  this  area  are  due  to  the  likelihood  that  mental  models  are 
dynamic  entities  that  can  have  a  multiplicity  of  forms,  even  for 
a  particulai*  individual  in  a  specific  situation.  Beyond  this 
issue,  other  types  of  limit  may  be  more  fundamental.  The  biases 
imposed  by  scientists'  own  mental  models  and  the  possibility  of 
an  uncertainty  principle  have  been  suggested  as  fundamental  in 
nature.  All  of  the  limits  outlined  in  this  paper  have  practical 
implications.  For  example,  the  design  of  "expert  systems"  is 
premised  on  humans'  abilities  to  verbalize  their  models;  in 
light  of  the  above  discussion,  this  ability  would  appear  to  be 
more  limited  than  is  commonly  assumed. 

Despite  the  fundamental  nature  of  some  of  the  limits 
outlined  in  this  paper,  the  issues  underlying  the  mental  models 
construct  are  important  and  deserve  substantial  attention.  What 
is  needed,  however,  is  to  move  away  from  the  perception  that 
"truth"  is  being  sought  and,  instead,  emphasize  the  utility  of 
researching  these  issues  to  advance  the  state  of  understanding  of 
learning,  problem  solving,  etc.  This  shift  should  help  to 
eliminate  many  minor  issues,  most  of  which  appear  to  emanate  f  ’om 
a  rather  zealous  tendancy  to  coin  new  terminology. 

By  purging  the  debate  of  these  minor  issues,  research  should 
be  able  to  focus  on  the  major,  substantive  issues  including 
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accessibility,  form  and  content  of  representation,  nature  of 
expertise,  cue  utilization,  and,  of  most  importance, 
instructional,  issues.  The  literature  is  replete  with  insightful 
thinking  on  these  issues  and  a  variety  of  interesting  and 
potentially  important  hypotheses  have  been  suggested. 
Unfortunately,  however,  there  is  a  paucity  of  solid  emirical  data 
available  to  support  or  refute  these  hypotheses.  At  the  moment, 
the  research  community's  ability  to  generate  conjectures  and 
publish  them  seems  to  be  much  greater  than  its  ability  to  test 
them  empirically.  What  is  needed  are  innovative  (and  validated) 
empirical  approaches  to  employing  the  mental  models  construct 
usefully,  most  likely  Involving  a  mix  of  several  traditional 
experimental  methods  with  newer  methods  such  as  computational 
modeling  and  linguistic  analysis. 

To  conclude,  the  search  for  mental  models  is  potentially  of 
great  importance:  any  success  that  is  achieved  is  likely  to  have 
substantial  impacts  on  system  design,  training,  etc.  However, 
there  are  fundamental  limits  on  what  can  be  clearly  seen  on 
looking  into  the  black  box.  It  appears  that  these  limits  will 
have  to  be  accepted  as  precluding  the  uncovering  of  "truth." 
Fortunately,  truth  may  not  be  necessary.  If  a  pragmatic 
perspective  is  adopted,  research  on  mental  models  can  avoid  the 
ephemeral  issues  and  concentrate  on  providing  rigorously  tested 
answers  to  a  variety  of  far-reaching  and  important  questions. 
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